The present invention relates to image processing systems for microscopes or other optical systems and, more particularly, to image processing systems which compose color images from low-cost optical systems using multiple wavelength illumination.
Traditional high performance wide-field microscopes 10 as shown in FIG. 1, achieve a high resolution color image of a specimen (object) 16 with a wide field of view of the object information 18. Also, by means of a beam splitter 24, the image 22, 26, 30 is fed simultaneously to both a video camera 32 and to an eyepiece 28. The focusing 48, 52 is often performed using a manual focus controller 46. However, autofocus controllers 42 which use the digital image context or additional optics can also be employed to send focusing commands 44 through a selector 50 to control the focusing mechanism 54.
With such microscopes, a user can look: (1) through the eyepiece 28; (2) at a video monitor 40 connected to the camera 32; or (3) at a computer monitor 70 connected through a frame grabber 62 and an image processor 66 to the camera 32. However, looking through an eyepiece strains the eyes, neck, and shoulders of the user. In addition, when two or more persons wish to view an image simultaneously, viewing the image through a monitor 40, 70 is clearly superior. Therefore, a high quality image on a monitor screen is preferable. Further, a computer 60, in addition to grabbing, storing, processing and presenting images, can use the output 34 from a camera 32 for image analysis.
However, the complexity and cost of such traditional high performance wide-field microscope systems is considerable. For example, such traditional microscopes provide illumination 14 using a white light source 12 such as a filament bulb. Such bulbs are not energy efficient. In addition, the color temperature of the bulb, which affects the color balance in the image, changes as the bulb ages.
Another costly requirement of traditional microscopes is that to get a good quality electronic image, one usually employs a relatively expensive three-chip RGB (red, green, blue) camera 32. Such a camera has internal prisms and filters to separate colors, one color for each of three black and white sensors. However, maintaining the relative positions and orientations of the prisms, filters and sensors during the lifetime of the camera is complex.
Also, for traditional microscopes, the most complex component is the objective (lens) 20 between the bulb 12 and the camera 32. Since the bulb provides white light, a traditional objective 20 is usually color compensated to supply an image for an eyepiece. In other words, the objective 20 has to produce reasonably sharp images 22 for the whole spectrum of interest at the same time. This spectrum is typically the entire visible spectrum.
To achieve the requirements for a microscope, the color compensated wide field objective 20 of a traditional microscope 10 is a compromise to achieve: (1) color compensation; (2) wide field of view; (3) magnification of 50-100 times; and (4) a numerical aperture which typically is 0.9 for a dry (air immersion) objective and 1.3 for an oil immersion objective.
To provide the compromise for a variety of conditions, traditional microscope systems are often equipped with a number of different objectives, for example, to adapt to the thickness of an optional glass cover slip or to provide overview images using a low magnifying objective. Such different objectives require the individual lens elements to be aligned with tight tolerances.
Also, even with the best possible objectives for visual light, some details of the image remain just beyond visibility. Therefore, a common image processing step is xe2x80x98image sharpeningxe2x80x99 of captured images 64 performed by an image processor 66. For example, by amplifying the higher spatial frequencies in an image, small details and edges become enhanced in the sharpened image 68. Selection of such frequency dependent amplification for the sharpening filter can be optimized if the characteristics of the objective are known. However, the objective characteristics are wavelength dependent, and therefore, ideally, each wavelength should have a respective sharpening filter.
Unfortunately, such separate sharpening filters are difficult to achieve with an RGB camera because each color component of the camera responds to light with a spectrum width of typically +/xe2x88x9250 nanometers which causes overlap between the colors. Thus, for example, the green component may respond to light with wavelengths extending from approximately 500 to 600 nanometers. Accordingly, once a sensor of the camera 32 has been exposed to white light, each wavelength""s contribution to the blurring of the image caused by the objective can not be determined and therefore the characteristics for a sharpening filter optimized for a wavelength can not be determined. Thus, a sharpening filter for such a system can only be a compromise.
Accordingly, traditional microscopes fail to provide a low cost system having a computer which generates high performance digital microscope images without filament bulbs and three-chip cameras. Such traditional microscopes also fail to permit the use of a simple objective which is suitable for wavelength tailored image sharpening filters.
This invention provides a novel design for a low-cost system for digital image microscopy. The system has a performance comparable to that of a much more expensive traditional microscope. Also, the invention can be used to improve the performance of an existing microscope system.
One object of the invention is to generate high quality microscope images on a computer screen to obviate any need of the user to look through the eyepiece. Accordingly, the invention does not provide a direct optical path from the specimen to a user""s eyes, but instead lets the images pass through a camera and an image processing computer.
Another object of the invention is to provide an optical system having: (1) an objective which receives electromagnetic radiation from an object, modifies the electromagnetic radiation, and emits the modified electromagnetic radiation as an image of the object; (2) a focusing mechanism which controls the movement of the objective along at least one path to modify the image; (3) one or more cameras which detect separate images for each of a plurality of frequency bands of the modified electromagnetic radiation of the image emitted by the objective; and (4) an automatic focus controller which, in accordance with the detected images: (a) provides control parameters to the focusing mechanism; and (b) determines for each of the frequency bands, an optimal image which corresponds to an optimal focus of the objective for that frequency band. The images can be detected/captured at different times. Also, the camera can be one or more black and white cameras.
A further object of the invention is to provide an automatic focus controller having: (1) a filter calculator for receiving the detected images as image signals and for generating filtered image signals such that noise components of the image signals have been reduced, the noise components being reduced by increasing energy contributions from parts of the image signals which contribute a relatively larger proportion to image components than the noise components and by decreasing energy contributions from other parts of the image signals which contribute a relatively larger portion to the noise components than to the image components; (2) an energy calculator for receiving the filtered image signals and determining energy levels of the filtered image signals; and (3) a control calculator for receiving the energy levels and for generating the control parameters in accordance with the energy levels.
An additional object of the invention is to provide a registration controller which aligns a plurality of optimal images. This registration controller can also have: (1) a transformer for receiving a first optimal image and a transformation, and for generating a transformed image, the transformation capable of translating, rotating, and/or magnifying the first optimal image; (2) a compositor for combining a second optimal image and the transformed image to generate a composite image; (3) an energy calculator for receiving the composite image and for determining an energy level of the composite image; and (4) a transformation generator for receiving the energy level and for generating the transformation in accordance with the energy level such that the transformation selected corresponds to a focus for the composite image.
A further object of the invention is to provide one or more image sharpening filters, each filter optimized for a particular frequency band.
Another object of the invention is to provide a source of electromagnetic radiation capable of emitting electromagnetic radiation in different frequency bands and of selectively emitting electromagnetic radiation from only one of the frequency bands. This source can include a plurality of separate sources, each separate source corresponding to one or more of the different frequency bands.
An additional object of the invention is to select: (1) one or more of the frequency bands to produce a response only from respective portions of the object; and/or (2) at least three of the frequency bands to correspond to red, green, and blue color components of visible light.
A further object of the invention is to provide an objective which: (1) lacks significant color compensation; (2) is selected such that the optimal focus for each frequency band occurs at a different position along the path of the objective; (3) provides optimal focus positions which are monotonically related to the frequency bands; and/or (4) is moved in a single unidirectional movement of the objective along the path to reveal the optimal focus for each of the frequency bands.
Also, an object of the invention is to provide a converter for transforming each n by 1 pixel of a composite image using a weight matrix to generate a respective 3 by 1 pixel for an RGB image, where n is the number of frequency bands. Additionally, the RGB image can be generated to simulate cameras which are different than the camera or cameras employed in the optical system.
Another object of the invention is to provide a method for operating an optical system having the steps of: (1) illuminating an object with electromagnetic radiation from a source; (2) modifying electromagnetic radiation from the object through an objective to form an image of the object; (3) controlling the movement of the objective along at least one path to modify the image; (4) detecting separate images for each of a plurality of frequency bands of the electromagnetic radiation; (5) providing control parameters for controlling the movement of the objective; and (6) determining from the detected images for each of the frequency bands, an optimal image which corresponds to an optimal focus of the objective for that frequency band. This method can also include steps of: (1) moving the objective only in one direction to reveal the optimal focus for each of the frequency bands; (2) sharpening one or more of the optimal images using a filter optimized for the respective frequency band; and/or (3) aligning two or more of the optimal images which each other.
These objects and other objects, advantages and features of the invention will become apparent to those skilled in the art upon consideration of the following description of the invention.